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Mapping the Reading Improvement Sector in New York City The Role of 
External Support Providers in Improving K-3 Reading Outcomes 


Abstract 

This report shares the results of a project designed to help build the collective capacity and increase the impact 
of the external support providers working to improve K-3 reading outcomes in New York City public 
elementary schools. In the first phase of the project, we identified all the programs in what we call the K-3 
reading improvement sector in NYC 2014-15. In the second phase, we examined the extent to which a sample 
of these programs have the goals, resources, and personnel to improve reading outcomes system-wide. In the 
third phase, we mapped the relationships among a sample of programs in the sector in 2016-17, the sources 
they rely on to support their work, and the NYC schools with whom they partner. Making these relationships 
visible shows the extent to which students from different backgrounds and schools can get access to 
information, resources, and expertise, and the extent to which programs are in a position to increase their 
collective impact through coordination and collaboration. 


Among the findings: 


« Over 100 programs are working in the K-3 Reading Improvement Sector in NYC 

« The sample programs in the sector focus on a wide range of reading-related goals, but a limited number 
of programs have demonstrated effectiveness 

« Twenty-six sample programs are connected to 161 different schools comprising 16% of all elementary 
schools in NYC (including 28% of the elementary schools in the Bronx and 26% of the elementary 
schools in Manhattan); and the programs are partnering with schools with relatively high levels of need 
in terms of both performance and poverty 

¢ Just over half of the sample programs describe themselves as collaborating or partnering with at least 
one other sample program, but almost half were not in regular contact with any other sample program 

« Sample programs received support from $7 different funders and 75 different sources for literacy 
expertise with little overlap 


‘These results suggest that sector programs have the goals, services, and personnel that could help improve K-3 
reading outcomes in New York City; they have the connections to share resources and expertise with a large 
percentage of elementary schools; and several clusters of connected programs could serve as a powerful force 
for increased focus and collaboration in reading improvement across the city. 


However, the collective impact of the sector suffers from the evidence that goals vary considerably. Student 
and teacher programs differ in terms of their goals and personnel, and only a few programs have had formal 
outside evaluations completed. In addition, many of the sample programs in the sector are working in 
isolation from other sample programs and are informed by a wide range of sources of funding and expertise 
that are themselves likely to be only loosely connected. Although the clusters of collaborating and frequently 
connected programs could serve as a basis for expansion within the sector, the unconnected programs and the 
disparate sources of funding and expertise suggest that explicit strategies will need to be developed to support 
greater coherence in the sector and to increase the effectiveness of the sector overall. 
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Executive Summary 


This report shares the results of a project designed to help build the collective capacity and 
increase the impact of the external support providers working to improve K-3 reading outcomes 
in New York City public elementary schools. In the first phase of the project, we identified all the 
programs in what we call the K-3 reading improvement sector in NYC 2014-15. In the second 
phase, we examined the extent to which a sample of these programs have the goals, resources, 
and personnel to improve reading outcomes system-wide. In the third phase, we mapped the 
relationships among a sample of programs in the sector in 2016-17, the sources they rely on to 
support their work, and the NYC schools with whom they partner. Making these relationships 
visible shows the extent to which students from different backgrounds and schools can get 
access to information, resources, and expertise, and the extent to which programs are in a 
position to increase their collective impact through coordination and collaboration. 


Among the findings: 
m Over 100 programs are working in the K-3 Reading Improvement Sector in NYC 


m Thesample programs in the sector focus on a wide range of reading-related goals, but 
a limited number of programs have demonstrated effectiveness 


m Twenty-six sample programs are connected to 161 different schools comprising 16% of 
all elementary schools in NYC (including 28% of the elementary schools in the Bronx and 
26% of the elementary schools in Manhattan); and the programs are partnering with 
schools with relatively high levels of need in terms of both performance and poverty 


m Just over half of the sample programs describe themselves as collaborating or 
partnering with at least one other sample program, but almost half were not in regular 
contact with any other sample program 


m Sample programs received support from 57 different funders and 75 different sources for 
literacy expertise with little overlap 


These results suggest that sector programs have the goals, services, and personnel that could 
help improve K-3 reading outcomes in New York City; they have the connections to share 
resources and expertise with a large percentage of elementary schools; and several clusters of 
connected programs could serve as a powerful force for increased focus and collaboration in 
reading improvement across the city. 


However, the collective impact of the sector suffers from the evidence that goals vary 
considerably. Student and teacher programs differ in terms of their goals and personnel, and 
only a few programs have had formal outside evaluations completed. In addition, many of 

the sample programs in the sector are working in isolation from other sample programs and 

are informed by a wide range of sources of funding and expertise that are themselves likely to 
be only loosely connected. Although the clusters of collaborating and frequently connected 
programs could serve as a basis for expansion within the sector, the unconnected programs and 
the disparate sources of funding and expertise suggest that explicit strategies will need to be 
developed to support greater coherence in the sector and to increase the effectiveness of the 
sector overall. 
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Introduction 


Over the past 30 years, an entirely new industry of “intermediary organizations” and “support 
providers” has emerged to help schools and districts improve (Hatch, 2002; Honig, 2004; Rowan, 
2002). These support providers include those that focus on creating new schools or turning 
around old ones (for example organizations such as Success for All and Expeditionary Learning) 
as well as others that work directly with students, teachers, or both to improve instruction and 
outcomes in particular subject areas like reading (such as Reading Recovery). Illustrating the 
growth of this industry, 


- . : in 2001, the Catalog of School Reform Models listed 63 different 
Anew industry of intermediary organizations focused on school improvement with 48 (76%) 
organizations and support providers has established after 1980 and only one in existence before 1960 
: . Northwest Regional Educational Laboratory, 2001). In 2018, a 
emerged to help schools & districts a of 13 mae websites revealed 151 of {ee ee 
Improve K-3 reading outcomes. focused primarily on “turning around” low-performing schools 
Meyers & VanGronigen (2018). 


Philanthropic initiatives as well as a series of federal policies have fueled both the demand 
for these kinds of organizations and the supply. For example, the New American Schools 
Development Corporation's break the mold design competition (NAS, 1991), the Annenberg 
Challenge, the Comprehensive School Reform Demonstration Program (CSRD, 1998), and the 
Investing in Innovation (i3) of the American Recovery and Reinvestment Act (ARRA, 2009) all 
offered funding that contributed to the development of a wide range of support providers. 
Policies including CSRD, No Child Left Behind (2002), and the School Improvement Grants 
program of ARRA (2009) have also required low-performing schools and districts to make 
improvements and encouraged them to draw on external support or approaches that have 
“proven results” in doing so. 


These initiatives and policies build on two key ideas: First, that, on their own, many schools do 
not have the capacity they need to make sufficient improvements in students’ learning and 
development (Goertz, Floden & O'Day, 1995; Hatch, 2013); and second, that there are “external 
support providers” that do have the capacity — the technical, human, and social capital - that 
can help large numbers of schools make those improvements (Datnow & Honig, 2008; Newman, 
King & Youngs, 2000). Technical capital refers to money and material resources; human capital 
refers to the skills, Knowledge and expertise of individuals and groups; and social capital 

refers to the access to resources and expertise provided by social relationships and networks 
(Corcoran & Goertz, 1995; Putnam, 2002; Spillane, Hallett, & Diamond, 2003). For example, 
external support providers can offer access to instructional materials and other resources and 
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trained and expert personnel that many school districts may not have or be able to develop 
on their own (Gamoran et al., 2003; Coburn et al, 2008). Furthermore, as “intermediaries” these 
programs may offer access to information, expertise and networks that can serve as a bridge 
between schools and researchers, policymakers and others (Coburn, 2005; Honig, 2009; Kerr et. 
al., 2006). 


Despite the benefits that support providers can bring to schools, evaluations show that, in 
practice, the overall impact of support providers is offen unclear or, at best, limited (Burch, 
2007; Munoz, Potter, & Ross, 2008; Rowan, 2008). Furthermore, in many cases, multiple support 
providers work in an uncoordinated fashion, which can actually hinder rather than help school 
and district efforts to build their own capacity (Hatch, 2002; Park & Datnow, 2008). Thus, external 
support providers that are disconnected from one another and isolated are likely to operate 
inefficiently and with redundancies and gaps in service. 


These findings call into question the basic assumption that external support providers will make 

it possible for large numbers of schools and districts to develop the capacity to reach student 
outcomes that they have not yet been able to reach on their own. Several steps could be taken 
to address these problems, however. For example, further investments could expand the number 
and reach of effective support providers. More coordinated, strategic efforts to connect 
providers and schools could increase the chances of success, particularly in targeted areas 

like reading and in districts where schools have access to significant external resources and 
expertise. This kind of increased access to materials, people, and relationships within and across 
organizations can facilitate joint problem solving, innovation, and organizational performance 
(French, 2001; Honig, 2004; Supovitz, 2008; Kania & Kramer, 2011; Reckhow, 2012). Connecting 
external support providers makes it possible for them to share resources and expertise, 
coordinate the deployment of their resources, and share information that can facilitate 

service delivery and the development of new tools and strategies. For schools that serve large 
concentrations of students living in poverty, access to support providers that are part of a robust 
social network may be particularly crucial because these programs can serve as gatekeepers to 
the information, resources, expertise, and power that contribute to more equitable outcomes for 
all students (Reckhow, 2012). 


Determining the most appropriate next steps to improve the effectiveness of support providers, 
however, depends on developing a much better sense of the number, goals, strategies, and 
connections among those providers currently at work in a given location. To fill this gap, 

with the support of the New York Community Trust and the Brooke Astor Fund for New York 

City Education, we launched a three-phase project to assess the collective capacity of the 
programs working in New York City public elementary schools to improve K-3 reading outcomes. 


We assessed the collective capacity of this K-3 reading improvement sector by addressing four 
questions: 


1. How many programs focus on improving reading outcomes at the K-3 level 
in New York City public elementary schools? 


2. To what extent do these programs have goals, resources, and personnel that 
can contribute to improved K-3 reading outcomes? 


3. Which New York City public elementary schools have access to the resources 
and support of the sector? 


1. To what extent are the programs in the sector positioned to increase their 
collective impact in the future? 


This report presents the context, background, findings, and implications from all three phases of 
the project. 
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Methods 


Context 


Reading at the elementary level is an especially productive area for external support because 
there is some consensus on approaches to instruction that are supported by research and at 
least some programs and approaches that have been identified as effective (e.g. Duke et. al., 
2011; Slavin et. al., 2009). New York City is also a particularly interesting place in which to carry 
out such a study of the collective capacity of reading improvement programs. Historically, New 
York City education policies have supported the development of external support providers. 

In particular, the Children’s First initiatives during mayor Michael Bloomberg's administration 
from 2001-2013 relied on the development of external support providers (O'Day, Bitter, & 
Gomez, 2011). The current administration of Bill De Blasio has eliminated some of the previously 
established supports for external providers, but schools and principals continue to have the 
authority to make their own selections of reading programs and other external partners. 

Lastly, a fund set up through the estate of the late Brooke Astor and overseen by the New York 
Community Trust has also invested $21.4 million to help 18 nonprofit support providers work with 
New York City public elementary schools to improve K-3 reading outcomes. The Brooke Astor 
Fund for New York City Education also provided the funds to carry out this study and produce 
this report. 


Data collection and analysis 


The three phases of this project were designed to explore the technical, human, and social 
capital of the programs working in the K-3 reading improvement sector in New York City. The first 
phase of this project identified all of the programs working in the sector in 2014-15. The second 
phase examined the extent to which a sample of programs at work in the sector in 2014-15 have 
the goals, resources, and personnel — the technical and human capital - needed to improve 
K-3 reading outcomes across New York City. In 2016-17, the third phase of this project mapped 
the relationships among a sample of sector programs, the sources they rely on to support their 
work, and the NYC schools with whom they partner. This work in the third phase provides a look 
at the social capital in the sector by making visible the relationships between the programs, 
their sources of support, and the schools with which they partner. The mapping of these social 
networks shows to what extent students from different backgrounds and schools have access to 
valuable information, resources, and expertise, and reveals the extent to which sector programs 
are in a position to increase their collective impact through coordination and collaboration. For 
the three phases of the project, we collected survey and interview data in a mixed-methods 
sequential explanatory design (lvankova, Creswell, & Stick, 2006), 


Phase I: Identifying programs in the sector (2014-2015) 


To determine which programs were working in the reading improvement sector in New York City 
in the 2014-15 school year, we established four criteria. The program: 


m Provides direct services to teachers, students, or both 

m Focuses on improving reading 

m Works with students or teachers from at least one grade at the K-3 level 
| 


Takes place in one or more New York City public school 


In order to find programs that might meet these criteria, we reviewed six sources: 


@ The list of applicants for an open call for proposals in 2013-14 for external support 
providers seeking to receive funding to work on improving reading outcomes in NYC 
elementary schools 


m Grantee lists of the top 15 grant-awarding foundations in 2013-14 
The New York State Department of Education approved supplemental providers list 


The NYC Department of Education’s list of organizations eligible for contracts with 
schools 


mg The What Works Clearinghouse 
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Nominations from literacy experts and NYC educators 


We used a saturated model approach to identify as many programs as possible that might 
meet these criteria (Marcja & Toffalori, 2012). This process yielded an initial list of 283 potential 
programs in the sector. We then employed an iterative crosschecking procedure to review 
publicly available program documents to determine whether each program on our initial 

list met all four criteria (Patton, 2002). At the end of this process, we identified 112 different 
programs that met our criteria for working in the K-3 reading improvement sector in New York 
City public schools in 2014-15. 


Phase II: Documenting the goals, strategies, resources, and expertise of sample 
programs (2014-15) 


As an in-depth analysis of all 112 programs was beyond the scope of the project, we sought to 
estimate the technical and human capital available in the sector by documenting the goals, 
strategies, resources, and expertise of a sample of those programs. To develop a representative 
sample of 32 programs out of the 112 programs, we used a two-step stratified random sampling 
procedure with proportional allocation based on a 95% confidence level with a margin of error 
of 15. First, to gain an overall sense of the nature and variety of the programs, we conducted 
an initial review of the program goals, approaches, and literacy terms used in the websites and 
public documents of all 112 programs. The initial review revealed differences in three aspects of 
the programs that seemed most likely to be associated with different approaches to reading: 


m Program Foci — (1) Reading; (2) Reading and Writing; (3) Reading and Other (e.g. 
literacy connections to art, content areas, or socio-emotional development) 


m Program Type — (1) Direct Service to Teachers; (2) Direct Service to Students; (3) Direct 
Service to Both Teachers and Students 


m Program Timing (for “student” and “both” programs) — (1) In-School: (2) Outside of 
School; (3) In- and Outside of School 


Second, we randomly selected programs from each of these sampling categories proportional 
to the overall sector. If a program declined to participate or was unreachable after multiple 
attempts, we replaced it with a program from the same sampling category. 


Once the sample was established, we reached out to representatives from each selected 
program to request a survey and interview. Our first aim was to identify someone at each 
program knowledgeable about the overall goals, strategies, and service provided for each 
program. Given that programs ranged substantially in size, the exact personnel differed. 
However, in most cases, an executive director or assistant, or the program manager for the 
specific reading program agreed to participate. 


Each program representative was then asked to complete a five-minute online survey 

prior to a 30- to 45-minute phone interview. The survey asked for basic descriptive program 
information such as program timing; dosage and duration; type and focus of services 

provided; and the number of schools with whom the program worked on an ongoing basis. 

The interview addressed in greater detail the reading program's background, goals, strategies, 
and approaches, such as what problem the programs tries to address and the focus of the 
program as it relates to reading and/or literacy. Other questions centered on the program's 
personnel, methods for assessing/evaluating the success of the program, and school partnership 
information. We also collected relevant documents such as publicly available annual reports, 
program descriptions, and program resources that provided further detail on the nature of each 
reading program. 


To analyze the surveys, interviews, and documents, we drew on our research questions to 
develop a coding tree focusing on the goals, assessments, and personnel qualifications of 

the programs. We also looked for terms and references related to reading instruction and 
educational approaches in general across the program materials and interviews. Examples 

of codes used included Common Core, comprehension, balanced literacy, reading at grade 
level, differentiation, and cultural diversity. The software program Atlas.ti was used to facilitate 
this process and to establish intercoder reliability (Tinsley & Weiss, 2000), as the coded data were 
crosschecked through a systemic process of codebook revision and recoding. 
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After we coded the surveys, interviews, and materials of each sample program, we looked 

for trends and patterns in terms of the nature of the goals, assessments, and personnel across 
the programs and looked for similarities and differences among the different categories of 
programs (among the programs focusing on students, on teachers, and on both, for example). 
To facilitate this analysis, we drew from research that has identified characteristics of policies 
that are likely to lead to improved outcomes in practice (Finnigan et. al., 2009; Porter et. al., 
1988). This work suggests that policies that are specific and consistent are more likely to be 
effective. Correspondingly, we noted the extent to which the goals, materials, personnel, and 
assessments of the programs were linked to specific or general goals, strategies, or outcomes 
related to reading. We then looked at the extent to which there is a consistent reading-related 
focus across programs and categories. 


Phase III: Mapping the social connections and sources of funding and 
expertise of a sample of programs (2016-2017). 


Phase Ill assessed the social capital of the sector by looking at the social network connecting a 
sample of programs in the sector in 2016-17 and their partner schools; the network connecting 
the sample programs themselves; and the networks connecting the sample programs with 
their sources of funding and the sample programs with sources of expertise in literacy and 
professional development. Although it would have been preferable to include all the programs 
working in the sector in this analysis, the demands for collecting this data on relationships from 
over 100 programs exceeded time, personnel, and resources available. 


Therefore, to collect the relationship data on a sample of programs in 2016-17, we first reviewed 
the list of programs working in the sector in 2015 and determined that four of the 112 programs 
were no longer working in New York City public elementary schools. We then sought to develop 
a representative sample of the 108 programs working at that time that was as consistent as 
possible with the initial sample in Phase Il. To that end, we contacted all of the participants 
from Phase I, as well as new programs following the same protocols as in the original sampling 
procedure in Phase Il. Of the 32 sample programs from Phase Il, 21 programs also participated 
in Phase Ill. Of those that did not participate, two programs were no longer working in New 
York City elementary schools; five programs did not respond to us within our data collection 
period, and four programs declined to participate in the third phase. These programs reflected 
the range of those in the sample and, therefore, were replaced by programs from the same 
sampling categories, contributing to a total of 30 participants for the Phase Ill sample. 


Social network surveys. To examine the relationships among the programs, the sources they drew 
on for support, and their school partners, we used social network analysis (Scott, 2017). To carry 
out the social network analysis, we first developed a social network survey that asked a lead 
staff member from each of the representative programs to indicate the frequency of interaction 
with the members of each of the other 108 programs in the sector during the previous year 
(never contacted; contacted once or twice; contacted every two to three months; contacted 
every month; contacted every week). In addition to asking about the frequency of contact, 

we also asked respondents to rate the strength of the relationships between programs (no 
interaction, communication, collaboration, and partnership) based on a scale adapted 

from Frey et al. (2006). We defined communication, or what Frey et al (2006) referred to as 
“coordination,” as the sharing of information and resources. We defined collaboration as 
“working together without a formal agreement,” and partnership as “working together formally 
to achieve common goals.” 


The survey also asked participants to indicate any organizations, people, or resources within or 
outside of the sector that they go to for: 

gm Funding 

m Literacy expertise 

m &xpertise in professional develooment 


These areas were selected because of their relationship to key elements of capacity-building 
and to the stability and expansion of the work of programs in the sector. 


Follow-up interviews. Interviews lasting approximately thirty minutes were conducted after 
program personnel had completed the social network survey. In the interview, we asked 
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program personnel to elaborate on the relationship with the other sector programs. Specifically, 
we asked with whom their program communicated, the format of their communication, and the 
content of their communication. We also asked interviewees to elaborate on the format and 
content of their relationships with the funders and sources of expertise they identified. To identify 
the school partners, the interview also asked respondents to provide a list of schools where their 
programs had an agreement to offer their services on a regular basis. 


Data analysis. As a first step in the analysis, we assessed the extent to which the programs were 
making the resources and expertise of the sector available to schools in general and schools 
most in need of support. To do so, we compared characteristics of schools partnered with one 
or more of the sample programs to the schools that had no partnerships with sample programs. 
We focused particularly on comparing the Economic Index of schools partnered with one or 
more sample programs to the Economic Index of schools with no partnerships with sample 
programs. (The Economic Index of schools is calculated using three numbers: the percentage of 
students in temporary housing, the percentage eligible for Health Reimbursement Arrangements 
(HRA), and the percentage eligible for free lunch.) We also looked at the average percentage 
of students achieving proficiency or advanced proficiency on the third-grade ELA test because 
of the emphasis in research and practice on ensuring that students can read by third grade. In 
this case, we compared the average ELA percentage of schools with sample program partners 
to the borough average. 


To analyze the data from the social network surveys, we used two-mode social network analysis 
to examine the relationships between the sample programs and: 1) the organizations they go to 
for funding; and 2) the groups and individuals they go to for expertise in reading, in developing 
literacy expertise, and in developing professional development expertise. Two-mode networks 
summarize the association between one entity and another, which can be a quick way of 
understanding the web of relationships. Programs were examined by degree: the number of 
links between an entity in one set and an entity in another. The subsequent interview focused on 
the nature of the interactions among programs and on the program's key sources of information 
and support. We then examined a series of network measures using UCINET software (Borgatti et 
al., 2002) to better understand the connections among sample programs, between programs 
and funders, and between programs and the sources of their expertise. To give a systemic 
overview of the social structure in the sector, we graphically represented network data in maps 
that make visible the connections through which information, resources, and expertise related 
to reading might flow. 


Limitations 


Our ability to identify programs in the sector depended on publicly available information. 
Therefore it is possible that there are programs at work in the K-3 reading improvement sector 
that provide little public information and that therefore are not captured in our lists. To mitigate 
this issue, we finalized the lists only after a review by a small group of reading experts and local 
educators who concluded that there were no programs they were aware of that were missing. 


The collection of data from the sample programs also depended on self-reports by individuals at 
each program who completed the survey and participated in the interview. Therefore, we also 
cross-checked that data with documents we collected from the programs (including, where 
possible, annual reports submitted to funders), and we shared the initial results of our data 
collection with the participating programs and distributed drafts of our findings and analysis with 
participating programs for feedback and to check for accuracy. 


Although we see it as a design choice rather than a limitation, analysis focused on the nature 
and specificity of the reading goals and reading-related strategies of the sample programs. 
Thus, this study does not take into account the extent to which these programs might support 
the achievement of other valued goals and purposes that go beyond conventional K-3 
reading outcomes. It is also worth noting that our criteria for program selection excluded some 
programs that could be viewed as part of the reading improvement sector. For example, we 
excluded those organizations that produce reading curricula but do not work directly with 
students or teachers because curricula in reading in particular have already been a subject of 
considerable research (Kragler & Martin, 2012; Bingham & Patton-Terry, 2013). We also excluded 
early literacy programs that work exclusively at the preschool level, as well as programs 

that operated independently of New York City public schools, such as many private tutoring 
programs. In short, this study illustrates one way, but not the only way, to conceptualize and 
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“map” the work among the many programs that could affect students’ reading outcomes at 
the K-3 level. 


Results 


How many programs focus on improving reading outcomes at the K-3 level in 
New York City public elementary schools? 


In Phase |, we identified 112 programs that were working in the K-3 reading improvement sector 
in New York City in 2014-15. These programs were operated by a variety of different kinds of 
organizations including local/community-based nonprofits, university affiliates, and consulting 
companies. Of the 112 programs, 49 (44%) focused specifically on reading, 33 (29%) focused on 
both reading and writing, and 30 (27%) focused on other topics like the arts or socio-emotional 
learning along with reading. Fifty-eight programs (52%) provide professional development to 
teachers, 45 (40%) provide support directly to students, and nine (8%) work with both teachers 
and students. Of 54 programs that serve students or both teachers and students, 18 (33%) 
operated during school hours, 21 (39%) took place outside-of-school time, and 15 (28%) served 
students at both times. 


These programs varied considerably in the specific ways that they worked with students and/ 
or teachers. For example, one program provided customizable professional development 

and coaching to help teachers develop their skills in balanced-literacy instruction. Another 
program trained volunteers to implement a reading intervention program during school for 
low-performing students. A third program brought together a community organization and a 
local university to pilot a storytelling curriculum for English Language Learners in an after-school 
setting. 


The review of the websites and available program documents of the 112 programs in the 
sector revealed the use of 255 unique literacy terms, with many programs using multiple terms. 
Nonetheless, there was relatively little consistency across the programs in the terms used. 

Even the most frequently-used term — Common Core - was mentioned by less than half of the 
programs (48) and only thirteen programs specifically stated that they were aligned with the 
Common Core. In a few cases, programs highlighted a specific Common Core related goal or 
practice such as “close reading” (mentioned by eight programs), but most mentions referred 
to the Common Core in general. The specific literacy skills cited included comprehension (29 
programs), vocabulary (21), phonics (15), fluency (13), and phonemic awareness (13). These 
goals reflect those discussed in the report of the National Reading Panel (2000) and identified as 
the five essential components of reading instruction as defined in the Reading First program of 
the No Child Left Behind Act of 2001. 


To what extent do programs have goals, resources, and personnel that can 
contribute to improved K-3 reading outcomes? 


In order to assess the extent to which the sector has the technical, human, and social capital 
that can contribute to improved reading outcomes among a large number of schools, in 
Phase Il we examined the survey results, interviews, the mission and goal statements, program 
descriptions, and instructional materials of the 32 sample programs. Consistent with the broad 
examination of the goals of the 112 programs in the sector, the review of the sample programs 
also revealed a wide range of different reading foci and no single dominant approach. Almost 
half of the sample programs (13 or 41%) emphasized a specific aspect of reading highlighted 
by the National Reading Panel such as comprehension, but other programs described goals 
such as getting students to read at grade level (19%) or supporting standards-based reading 
instruction (see Figure 1). Figure 1. Reading goals by program type (%) 


Notable differences in the approaches of the programs serving students and the programs 
working with teachers also emerged. Four out of the six programs focused on Common Core 
reading standards, and all four of the programs focused on balanced literacy, were programs 
working with teachers. In contrast, five of the six programs focused on reading at grade level 
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Figure 1. Reading Goals by Program Type (percentage) 
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worked directly with students. In terms of specific skills, all three of the programs focused on 
phonics were student programs. In contrast, all three of the programs that centered on oral 
language worked with teachers or both teachers and students, while comprehension was 
a focus of two student programs, three teacher programs, and one program serving both 
teachers and students. 


The programs working with students, working with teachers, and working with both teachers 
and students also differed substantially in their overall approaches to reading instruction. For 
example, the 15 programs that worked with teachers typically provided a series of professional 
development workshops or coaching for individual teachers or both. The 13 programs working 
directly with students provided staff and/or volunteers who implemented a specific reading 
program or provided tutoring. The four programs that worked with both students and teachers 
(including three arts-based programs) developed their own curriculum or approach and 
provided trained staff or “artists-in-residence” who could work directly with students, but those 
staff members and artists also offered professional development workshops to help the school- 
based educators learn how to implement the program's approach. 


In terms of program dosage, sessions for the 13 student programs ranged from 30 to 90 minutes, 
with most lasting for a 45-minute class period. Six of the 13 programs for students provided 
regular (weekly or daily) services for a full school year, either in school or in an afterschool 
program ata partner school. Five programs provided their services both during the school year 
and over the summer. The other two programs worked with students for a shorter period of time. 


Six of the 19 programs that worked with teachers or both teachers and students provided weekly 
services to teachers over an entire school year. The other 13 programs provided a variety of 
services, such as individual coaching, online support, or modeling with the amount and type of 
support differing for each teacher served. 


Figure 2. Hiring Requirement by Program Type (%) 
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The review of the extent to which the programs provide schools with access to reading-related 
knowledge and expertise revealed another difference between the student and teacher 
programs. Programs working with teachers or with both teachers and students reported requiring 
new personnel to have more education and teaching experience than programs working 
directly with students (see Figure 2). 


In fact, every program working with teachers or working with both teachers and students 
required staff to have either a bachelor’s or a graduate degree, and all but one of the 
programs working directly with teachers required their staff members to have teaching 
experience. In contrast, three (23%) of the programs working directly with students reported no 
specific qualifications for staff and four (31%) only required a high school diploma. Only one of 
the 13 student programs required staff to have teaching experience. 


However, the results for training were almost the opposite. In fact, all of the programs working 
directly with students provided training for their staff members and eight (62%) required training 
specifically related to reading. Among the programs working with teachers or both teachers 
and students, just under a third provided training specifically related to reading (27%), and 
almost two thirds (66%) did not require any training (see Figure 3). 


Figure 3. Training Provided by Program Type (%) 
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We also reviewed the sample programs’ approaches to assessment and evaluation to ascertain 
the extent to which they had in place mechanisms for improving their performance and 

had external evaluations demonstrating their effectiveness. Of the 32 sample programs, 23 
programs (72%) employed assessments focused explicitly on students’ reading abilities, such 

as Fountas and Pinnell (2011) reading-level assessments or state ELA exam scores. We classified 
three programs (9%) as using other assessments because they did not directly assess students’ 
reading abilities. Two of these programs were programs that worked with teachers and used 
rubrics to assess instruction. The third program focused on reading and visual arts standards and 
assessed these standards through examinations of student work. Six programs (19%) did not use 
assessments for students or teachers. 


Of the 32 programs, six (19%) had publicly available evaluations carried out by external 
evaluators (not employees of the program), with results from two of these evaluations published 
in academic journals. Eight programs (25%) produced their own internal summative evaluation 
reports (usually shared with their funders). However, another eleven programs (34%) only carried 
out internal progress monitoring through formative assessments of student outcomes, and seven 
programs (22%) did not carry out formal evaluations or report ongoing internal monitoring of 
their outcomes at all (see Figure 4). 


Figure 4. Evaluation Approach Reported (%) 
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Figure 5. Geographic map of partner schools 
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Although these results show that less than a quarter of the programs produced external reports, 
a substantial majority, 23 programs (72%), employed assessments focused explicitly on students’ 
reading abilities. Two of those 23 programs relied on state ELA exam scores, while the other 21 
used performance assessments such as Fountas and Pinnell (2011) reading-level assessments, 
with many referring to state ELA exam scores as well. 


In sum, the results showed that teacher and student programs took somewhat different 
approaches to evaluation and assessment. In fact, the five programs that did not use any 
assessments were afterschool programs working directly with students; four of these programs 
also did not carry out any evaluations while one monitored progress internally. In addition, the 
four programs classified as using “other” assessments were teacher programs that focused on 
reading standards. Two of these programs worked with teachers and used rubrics to assess 
instruction in general. A third program focused on reading and visual arts standards and 
assessed these standards through examinations of student work. 
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Which New York City public elementary schools have access to the resources 
and support of the sector? 


To analyze which schools in New York City are getting access to the resources, services, and 
personnel of the programs in the sector, we reviewed the number and characteristics of schools 
the sample programs worked with in 2016-17. These analyses were based on data from the 26 
sample programs that provided complete lists of their school partners that year. Two programs 
did not provide data on specific school partners, and two reported partnering with so many 
schools that they could not list them alll. 


The 26 programs worked in 161 different schools across all five boroughs of New York City. That 
means these sample programs were working in about 16% of all the public elementary schools 

in New York City. The 161 partner schools are spread across all boroughs, but a much larger 
percentage of schools in the Bronx (28%) were partnered with sample programs. Manhattan had 
the next highest percentage (26%), and Staten Island the third highest (16%). Even though the 
largest number of schools overall are in Brooklyn (10%) and Queens (9%), they had the lowest 
percentages of partner schools (see Figure 5). The significant majority of partner schools (75%) 
worked with only one of the sample programs, but 40 schools worked with two sample programs 
or more. 


Overall, the programs in the sector appear to be sharing their resources and expertise with 
schools with lower-performing students and with schools with relatively high rates of students 
living in poverty as measured by the school’s Economic Index . However, there are also 
indications that different kinds of schools are getting access to different kinds of resources 

and expertise. For the most part, schools that partner with multiple programs or with student 
programs have a lower mean percentage of students passing New York’s English Language Arts 
exam in third grade, and they have higher levels of economic need than schools that partner 
with teacher programs and programs serving both students and teachers (see Figure 6). A 
statistical comparison (one-way ANOVA) shows that the percentage of third-grade students who 
pass New York's 


Figure 6. School Partners Mean % Economic Index and % Failing to Reach Proficiency 
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English Language Arts exam is significantly lower in schools with multiple program partners 
(M=27.88, SD= 12.86, p = .000) and in schools with one student partner (M=27.40, SD=13.51, p = 
.000) than in schools with one teacher program partner (M=42.79, SD=21.44, p = .000) or with 

a program partner that works with both teachers and students (M=45.33, SD=17.01, p = .003). 

In addition, the average poverty rate is significantly higher in schools with multiple program 
partners (M=83.58, SD= 7.85, p = .000) than in schools that partner with one teacher program 
(M=68.85, SD= 21.76, p = .000) or in schools that partner with a program that works with both 
teachers and students (M=59.72, SD= 17.59, p = .000). Schools with one student program partner 
also have a significantly higher average poverty rate (M=75.96, SD= 13.13, p = .003) than schools 
with a partner that works with both teachers and students (M=59.72, SD=17.59, p=.003), but there 
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was no significant difference between the average poverty rate of schools with one student 
program partner and with one teacher program partner. 


To what extent are programs positioned to increase their collective impact in 
the future? 


To get a sense of the extent to which programs in the sector are positioned to share information 
with one another, to coordinate services, and to increase their collective impact, we explored 
three questions: 


m Howare the programs in the sector connected to one another? 

m Where do the programs get their funding? 

m What sources do the programs turn to for expertise? 
How are the programs in the sector connected to one another? The analysis of the social 
network surveys shows that 13 of the 30 sample programs (43%) were in contact with at least 
one other sample program on a monthly basis (monthly contact is generally considered 


“frequent contact” and serves as the standard for regular exchanges of information, resources, 
and expertise) (see Figure 7). 


Figure 7. Network of relationships between sample programs showing frequent contacts 
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The connections among many of those 13 connected programs were also reciprocal (each 
program named the other as a frequent contact). In general, these two-way reciprocal 
connections are considered to provide a strong foundation for the sharing of information, 
resources, and expertise. Further, the network of 13 connected programs included a number of 
programs that could serve as “hubs” and “brokers” that help share information and resources 
throughout a network. The larger red circles represent these hubs and brokers, with the size 
of the circle representing the number of times that program was listed as a frequent contact 
by other programs. Thus, the largest of these circles can be considered the most “popular” 
programs. Together, the number of popular programs and reciprocal ties suggest that the 
network among connected sample programs has a strong central core. However, 17 of 

the sample programs were not in frequent contact with any of the other sample programs, 
suggesting that they have limited access to the information and resources shared among the 
connected programs. 


The network maps also reveal several “sub-networks,” distinct groupings among different kinds 
of programs. In fact, 10 of the 13 sample programs that were in frequent contact are programs 
that provide direct services to students or direct services to both teachers and students. Only 
three of the programs that provide direct services to teachers were in frequent contact with 
other sample programs, and each of those three programs was only connected to one student 
program (a different student program in each case). No teacher programs were in frequent 
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contact with other teacher programs in the sample. In short, there was a strong network among 
a group of student programs, but many programs (including almost all teacher programs) were 
isolated from other programs in the sector. 


In addition to the frequency of contact, we also asked sample programs to identify which 
programs they collaborated with and those with whom they had a partnership. Collaboration 
was defined on the survey as “working together without a formal agreement,” and partnership 
was defined as “working together formally to achieve common goals.” In this case, 17 sample 
programs (57%) were reported to be in collaboration or in a partnership with at least one other 
program (see Figure 8). 


Figure 8. Network of program relationships reported among sample programs 
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Of these 17 programs, 13 were the same programs that reported being in frequent contact 
with at least one other sample program. In addition, 10 of those 13 (33% of the all the sample 
programs) were involved in multiple partnerships. Of those 10, seven were student programs, 
one was a teacher program, and two were programs serving both students and teachers. 
These results suggest that a slightly larger number of programs view themselves as working 
closely with other sample programs, but, again, almost half of the sample programs were not 
in collaboration or partnership with any other sample programs. Reinforcing the earlier results, 
student programs were much more likely to report that they were working in collaboration or 
partnership with other sample programs. Although five teacher programs reported working in 
collaboration or partnership, the majority of teacher programs were not working in collaboration 
or partnership with any other sample programs. 


We also looked at the program characteristics and our interview data to get a sense of the 
factors that might explain why some programs were more closely connected than others. 
Notably, we found that the programs cited most frequently as partners often participated in 
collaborative groups (for example, those organized by the Pinkerton Foundation or the New York 
City Council or those funded by the Brooke Astor Fund for New York City Education). 


Interviews with members of the programs involved in four or more partnerships also suggested 
that these programs partnered with those with whom they shared interests and strategies. For 
example, respondents from some of these programs noted that they were engaged in efforts 
to collaborate on after-school programs and some pointed to expanded-day initiatives and 
summer initiatives as reasons for forming partnerships. Respondents from two other programs 
noted that they were involved in collaborations involving community-based organizations 
(CBOs) with a variety of different goals. As one respondent stated, they saw the other CBOs as 
allies and had regular meetings where they communicated closely. 


In contrast, of the ten sample programs that were not frequently connected and were not in 
any collaboration or partnership, six were teacher programs. Respondents from these programs 
noted that the programs did not need the help or support of others. One respondent, for 
example, stated that they “don’t use any of these programs, because | have my own program, 
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so | don't use any of their services.” Another mentioned, “I don’t have any desire to connect 
with other organizations... | would be open to it if we could structure it in a way that would 
make sense for everybody... I'm not running up against problems that | can’t solve. | feel like I’m 
solving these problems that everybody says they have, and no one is listening.” 


Although obtaining surveys from all programs in the sector went beyond the scope of this 

study, the social network surveys with the sample programs asked them to use the same 

scale to describe the frequency of their interactions with each of the other 78 programs in 

the sector. These results of this analysis of “one-way relationships” showed that only nine of 

the thirty sample programs (30%) reported frequent contacts with at least one of the other 78 
programs in the sector. Those nine sample programs reported frequent contacts with a total of 
fourteen of the 78 other programs in the sector. Interestingly, a much larger number of sample 
programs (16 or 53%) reported being in collaborations or partnerships with one of the other 78 
programs in the sector. This percentage was almost the same as those reporting collaborations 
or partnerships with other sample programs. Of the thirteen sample programs that reported 

no frequent contacts or collaborations or partnerships with other sample programs, only three 
reported frequent contact or a collaboration or partnership with one of the other 78 programs 
in the sector. Thus, ten sample programs (33%) did not report any frequent connections or 
collaborative or partnership relationships with any of the other programs in the sector. Paralleling 
the findings of the relationships among the sample programs, the student programs in the 
sample were most likely to report frequent connections, collaborations or partnerships with other 
programs in the sector, while teacher programs were most likely to report no connections. These 
results need to be treated with caution, but they provide no evidence that the relationships 
among the sample programs are substantially different from their relationships with all of the 
other programs in the sector. 


Where do the programs get their funding? The sample programs get most of their funding from 
contracts with individual schools (7 programs) or from contributions, grants, or donations from 
charitable foundations, government agencies, and nonprofit organization (21 programs). Of the 
seven programs relying on contracts, four were for-profit organizations, and six were professional 
development providers. The 21 programs that relied on contributions and grants reported 
receiving funding from 57 different organizations (see Figure 9). 


Figure 9. Network of relationships between sample programs and charitable foundations, corporations, 
government agencies, and non-profit organizations 
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The majority of organizations named as funders by the sample programs were charitable 
foundations (64%), such as the Gates Foundation or the Ford Foundation. Another 17% of 
funders were corporations such as Bloomberg LP or Target. Government agencies, such as the 
Institute of Education Sciences and the New York City Department of Cultural Affairs comprised 
12% of the sample’s funders. Lastly, 4% of funders named were nonprofit organizations, such as 
Americorps or the United Way, who are often themselves funded by foundations, corporations, 
government grants, and private donations. 
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Of the 57 organizations named as funders, the vast majority (75%) supported only one program 
in the sample; 12% funded two sample programs; and a small handful of funders were named 
several times by sample programs. Those funders included the Brook Astor Fund for New York 
City Education (8 programs), the Pinkerton Foundation (6 programs), the US Department of 
Education and the NYC Department of Youth and Community Services (4 programs each); and 
the Hecksher Foundation for Children (3 programs). At the same time, on average, programs 

in this network reported having three different funders; responses varied substantially, however, 
with one program identifying 14 different funders. 


In interviews, a number of members of the sample programs described their funders as actively 
working to support their grantees beyond simply providing funding. Several respondents talked 
about their funders as wanting to be “thought partners who deeply want to understand what 
we're doing and how it’s going.” Others described their funders as wanting to have “high-touch 
relationships” and as “hands-on,” scheduling meetings, offering professional development 
sessions, and conducting site visits. Several interviewees also described ways in which their 
funders provided opportunities for collaborations and partnerships with other programs. For 
example, one respondent reported that one of their funders organized a professional learning 
community around school topics and invited a handful of similar programs to share expertise 
and learn from one another. Several interviewees also described how formal collaboratives with 
multiple programs helped to alleviate the pressures of competing for funds. One respondent 
said a funder asked the members of their collaborative, “if you didn’t have to think about 
money how would you all work together to address this problem of students not reading on 
grade level?” 


At the same time, interviewees also articulated a number of challenges associated with 

the competition for funding and related demands from funders. “One of the challenges,” a 
respondent declared, “is that we are all looking for business... whether you’re a nonprofit, or a 
for-profit, we all need work. That’s one thing that sometimes can get in the way of connecting 
or partnering.” In fact, interviewees from several of the teacher programs explicitly stated that 
their program will not work in a school if it is already working with another reading program. 


Even when the programs were working together in a collaborative, concerns about competition 
and funding can haunt the work. As one interviewee stated, “initially you consider these 
programs working in the same space to be competitors for the same funds, and it’s hard to 
change from a competitive mindset into a partnership mindset.” Another respondent whose 
programs was part of a collaborative explained, “in the next school year there will be a 
reduced financial award from the foundation, so we need to figure out as a group how to work 
together to fill that gap, and that will take our partnership to a new realm, and everyone's a bit 
skeptical and cautious... you don’t want to bring [a funder] out and get them fascinated in your 
partner’s program and fund them instead. That’s happened to [us] in the past.” 


Some interviewees also talked specifically about the challenges of meeting the stipulations 
and expectations of their funders, particularly if funding is only available through requests for 


Figure 10. Network of relationships between sample programs and sources of literacy expertise 
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proposals (RFPs). As one respondent put it, “we're kind of stretching ourselves to meet all of their 
compliance and regulations that comes with those strings.” Another mentioned that the specific 
focus of their program often makes securing funding a challenge, as they “haven't been able 
to find a funder to pitch this to because New York City has a different focus.” 


What sources do the programs turn to for expertise? Sample programs in the sector drew 

on extensive and disparate sources for literacy expertise and for expertise in professional 
development. In terms of literacy expertise, three programs (10%) reported that they did not 
seek out literacy expertise from any individuals or organizations, but the other 27 programs 
named 75 different sources that they drew upon for literacy expertise (see Figure 10). The 
number of sources named by each of these 27 programs ranged from one to eight. 


The sources of literacy expertise included professional organizations, university professors, 
consultants, websites, and conferences. University-affiliated faculty and organizations 
accounted for the largest category of literacy expertise, with 19 different University-related 
sources mentioned. Curriculum providers and curricular materials were the second-largest 
category of literacy expertise, mentioned 13 times. Finally, professional organizations and 
national/local service providers were mentioned 11 times each. 


Notably, only a few sources were named by more than one program. In fact, of the university- 
affiliated organizations - the largest category of literacy expertise - only two organizations, both 
of which offer professional development around literacy, were mentioned by more than one 
program. University-affiliated faculty were often connected with only one program, and, in 
interviews, program staff explained that university faculty typically served as a program advisor 
or a curriculum developer. 


A cluster of two student programs, three teacher programs, and three programs working with 
both students and teachers (seen on the right side of Figure 10) included links to a group of 
professional organizations. Those professional organizations included the National Council 

for Teachers of English (NCTE) (mentioned by four programs); the Association for Supervision 
and Curriculum Development (ASCD) (mentioned by two programs); and the International 
Literacy Association (ILA) (mentioned by two programs). The multiple mentions of professional 
organizations, and particularly the conferences that they organize, suggest that conference 
formats may be one of the most popular sources for expertise on literacy. The only remaining 
organizations with more than one mention were the NYC Department of Education and Engage 
NY (an online resource created by the New York State Department of Education). These were 
mentioned three and two times, respectively. 


Since many teacher programs provided literacy services in the form of professional 
development, we also asked programs whom they go to for expertise in professional 
development. Overall, 18 programs (60% of the sample) named 31 unique sources (see Figure 
11). University-affiliated individuals/organizations accounted for 26% (eight sources) of the 
professional development sources mentioned. Professional organizations, curriculum providers, 


Figure 11. Network of 30 sample programs showing frequent contacts (once a month or more) 
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and service providers each accounted for 23% of the mentions (seven organizations each). 
Finally, one national network and one foundation were mentioned (3% each). For the most 
part, the 31 sources of professional development expertise were named only once, and there 
were only four sources that were named more than once. Two teacher programs named NCTE 
and ILA (both professional organizations that also hold conferences) as a source. Student 
Achievement Partners, a curriculum provider, and a university professor, were also named as a 
source by two different programs. 


While programs serving teachers were much more likely to name a source of professional 
development expertise than student programs were, we found that a mix of both teacher and 
student programs reported drawing on sources of professional development expertise. Of the 
18 programs that named a source of professional development expertise, eight were teacher 
programs (44%), seven were student programs (39%), and three were programs serving both 
students and teachers (17%). However, 12 sample programs (40% of the sample) did not report a 
source of expertise at all and five of those were teacher programs, meaning 38% of all teacher 
programs did not name a source. Four (36%) of the student programs did not name a source, 
and three (50%) of the programs serving both students and teachers did not name a source. Of 
the teacher programs that did not name a source, a respondent from one program stated that 
they themselves were the source of professional development that they provided, and so they 
did not seek professional development expertise elsewhere. 


Summary of the relationships and sources among sample programs. The social network maps 
reveal for the first time the extent to which programs in the sector are connected with one 
another and the extent to which they are operating in independent and isolated ways. These 
findings suggest that in 2016-17 the sample programs formed three different groupings: a group 
of about half of the programs who were isolated; a group of programs that were in regular 
contact (monthly or more) with at least one other program; and a group of almost one third of 
the sample programs who describe themselves as working in collaboration or partnership with 
multiple programs and were making some explicit efforts to coordinate their work. 


The relatively tight connections among the programs that are frequently connected and 
working in collaboration or partnership suggest that some sector programs are well positioned 
to share information, expertise, and other resources. However, almost all of the most tightly 
connected programs were programs serving students and many of the programs that are 
isolated are teacher programs. That suggests that teacher programs are much more likely to be 
operating independently of other programs and have fewer opportunities to share information, 
expertise and other resources. 


Along with the limited connections among many programs in the sector, the fact that programs 
draw ona disparate array of sources of funding and expertise indicates a lack of coherence 
and consistency in the support offered to schools by the sector. Nonetheless, many of the most 
frequently connected programs participated in collaboratives supported by organizations like 
the Pinkerton Foundation and the Brooke Astor Fund for New York City Education; this suggests 
that explicit and direct support for programs to work together and collaborate could have a 
beneficial impact on the development of the social networks and the collective impact of the 
sector as a whole. 


Summary and Implications 


When we launched this study, there was no clear sense of how many programs might be 
working with New York City elementary schools to improve K-3 reading outcomes, and little 
was known about the nature or collective potential of their work. This analysis shows that there 
are over 100 different programs in New York City focused broadly on improving reading at the 
K-3 level. Our analysis of a sample of those programs shows that almost half of those programs 
hired personnel with at least a bachelor’s degree or provided training for their staff members 
who worked in schools. In addition, most programs use specific assessments related to reading 
outcomes and make some effort to evaluate their work or monitor their progress. Although only 
about a fifth of these programs have had external evaluations, most of these programs have 
in place or could develop mechanisms for assessing their effectiveness and improving their 
outcomes in the future. 
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In order for these support providers to have a collective impact on reading outcomes at any 
significant scale, they also need to have the relationships that enable them to bring their 
resources and expertise to a large number of schools. In this case, 26 of the sample programs 
were working in 16% of the public elementary schools in New York City. The sheer number of the 
sample programs already working in areas like the Bronx suggests that the programs working 

in the reading improvement sector have the reach to be a valuable lever for system-wide 
improvements in reading instruction. Furthermore, there are several clusters of programs that are 
frequently connected and working in collaborations or partnerships. These clusters could serve 
as a powerful force for focus and collaboration in reading improvement across the New York 
City. 


At the same time, to have a strong positive impact on K-3 reading outcomes across the sector, 
schools partnering with one of the sector programs should be able to access comparable 
resources and expertise. However, this analysis shows that there are some important differences 
in the nature of the programs, their expertise, and the way they are distributed among schools 
that have important implications for the collective capacity of the sector: 


m There is considerable variability in the goals, personnel, and approaches to assessment 
and evaluation of the programs overall. Notably, the programs show no consistency 
and varying levels of specificity in their reading goals. Instead, the language used in 
program descriptions and program documents reflects the extent to which policies 
change and policymakers promote different and sometimes conflicting approaches 
to reading instruction. For example, some of the programs emphasize specific skills like 
comprehension and vocabulary that echo the language championed in policies like 
the Reading First program of NCLB (2001) while others adopt the language of reading 
standards from the Common Core (National Governors Association, 2010). Thus, the 
shifts in priorities and policies may contribute to a lack of coherence in the sector, 
making it more difficult for programs to coordinate their work and to have focused, 
sustained, collective impact. 


m There are distinct differences between the programs working with students and the 
programs working with teachers. On the one hand, the student programs tend to be 
focused on goals like grade level reading and discreet skills like phonics in activities 
that outside staff and personnel bring into the schools. These programs depend on 
finding time during the school day or after school and/or during the summer when 
they can be added on to the reading instruction students get in school. Many of the 
student programs tend to be connected to groups and networks, providing their 
school partners with access to information and resources from other programs within 
the sector. On the other hand, teacher programs tend to focus on helping teachers to 
improve their overall instruction through approaches like balanced literacy or alignment 
with standards. These programs are designed to be integrated into students’ reading 
instruction during the regular school day, but also require professional development 
time with teachers. For the most part, the teacher programs are working independently, 
with at least some indicating that they specifically choose school partners that do not 
work with other programs. As a consequence of these differences, schools working with 
different kinds of programs get access to different kinds of resources and support. 


mg Different kinds of schools work with different kinds of programs. Overall, programs in the 
sector work with high percentages of schools that are lower performing and that serve 
students from families with high levels of poverty. However, the schools that partner with 
student programs or that partner with more than one program tend to have higher 
levels of need than schools that partner with teacher programs and programs serving 
both students and teachers. These findings suggest that schools in different locations 
and with different populations of students may be getting access to different kinds of 
resources, support, and expertise. Although it is too early to tell how significant these 
differences are and how they might impact schools, the implications for equity and for 
effectiveness warrant further examination. For example, schools that work with multiple 
programs may benefit from having access to more resources and approaches, but 
those schools may also encounter increased demands and challenges in coordinating 
the work of different programs. 


Taken together, the lack of consistency across the programs in terms of the goals and expertise 
and access they provide, and the fact that different elementary schools are likely to be getting 
access to different kinds and of reading-related support, limits the collective capacity of the 
sector. Essentially, schools have access to a variety of different goals and services that may 
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or may not be aligned with their needs and the demands they face, and they have limited 
information to help them decide which programs to work with. 


In addition to the variety of goals, the differences among the student and teacher programs, 
and the differences in access among partner schools, the collective impact of the sector also 
suffers from the evidence that the programs in the sector are informed by a wide range of 
sources of funding and expertise that are themselves likely to be only loosely connected. The 
unconnected programs and the disparate sources of funding and expertise mean that most of 
these programs have to demonstrate their effectiveness while working independently of other 
programs. The fact that only three of the teacher programs reported frequent contact with 
other sample programs, and that five teacher programs did not name a source of professional 
development expertise, also indicates that these programs may be operating even more 
independently than the student programs within the sector. These results show that many of the 
programs in the sector have to compete for funding and develop their resources and services 
with relatively limited information about what other programs are doing or where they are 
working, and with little or no information about the specific needs of schools. 


Recommendations 


Given these results, no single strategy will be sufficient to build the collective capacity of the 
sector moving forward. Instead, different approaches are needed to take advantage of the 
different strengths of the programs working with students and those working with teachers and 
to promote greater collective impact on reading outcomes across the city. 


Create an inventory and “map” that shows all the programs working in the sector and the 
schools with which they partner. Instituting a regular inventory of programs and “mapping” of 
the partner schools could identify gaps in services and underserved schools and neighborhoods. 
Schools and programs could also use this information to make more strategic choices about 
who to work with and to facilitate coordination when schools have multiple program partners. 


Identify common needs, assessments, and tools to promote coherence across the sector. The 
NYC Department of Education, an alliance of sector programs, or another local organization 
could establish common areas of focus or standards to help ensure the quality and consistency 
of work. Programs could continue to adaress different goals and needs but the shared 
understanding would facilitate alignment and consistency. In addition to articulating key 

local needs or goals related to elementary reading, productive developments might include 
standards for the “dosage” or extent of services support providers should offer; sharing of 
common reading assessments; identification of core practices for reading instruction; and 
creation of workshops for program personnel to support their use of common assessments and 
core practices. 


Foster strategic alliances among programs working with students and programs working with 
teachers. Productive collaborations among student and teacher programs could both deepen 
their work and expand their reach. “Matching” support from afterschool programs for students 
and in-school programs for teachers and/or students would be particularly effective, and the 
adoption of complementary goals and common assessments could help to magnify effects. 
Identifying the obstacles and opportunities for collaborations among a small number of 
complementary programs could also help to provide a foundation for more coordinated work in 
the future. 


Build broader coalitions for neighborhood impact. Schools can benefit from the support of 
sector programs, but many different people and institutions can help to create opportunities 
for students to learn to read that support collective impact in different communities and 
neighborhoods. In New York City, for example, East New York Reads (in Brooklyn) and the 
Neighborhood Literacy Initiative (in South Jamaica, Queens), already bring together some 
sector programs with local community organizations and institutions like the Brooklyn and 
Queens public libraries. 


Create a local “hub” for the sharing of information and expertise around reading instruction 
and reading improvement in New York City. Education leaders, policymakers, and funders 
can support coordination in the sector by creating an annual forum for educators, community 
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members, funders, and researchers. Such a forum could facilitate the development of 
relationships among programs, promote the sharing of information about reading research, and 
support reflection on overall progress and sector development. 


Investments in building the collective impact of the sector, however, should take into 

account that schools and districts that already have some capacity - a foundation of strong 
relationships, instructional coherence, internal accountability, or student performance that 

is already relatively strong — are most likely to benefit from external support (Hatch, 2009; 
Newman, King, & Youngs, 2000; Bryk & Schneider, 2002). Further, improvements may most likely 
be achieved when schools have the expertise to pick their external partners carefully and 
already have the capacity to adapt external services to match the needs of their students and 
staff (Datnow, 2000; Hatch, 2002). 


Under these circumstances, education leaders, policymakers, and funders have to balance 
efforts to promote coordination and coherence in the sector with the recognition that 
imposing narrow goals and limited sets of evaluation measures can also reduce flexibility and 
adaptions designed to meet local goals and needs of students and schools. In short, despite 
the assumptions that external support providers already have the resources and expertise 
that schools need to improve outcomes, strategies like these begin with the recognition that 
investments need to be made in building the capacity of both external support providers 
and schools; but they also establish a middle way between piling on bureaucratic controls, 
mandating collaboration, and letting “1000 flowers bloom.” 
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